1. Existence of social differences in health (Descriptive)
   

2. Causes of observed social differences in health (Etiologic)

   • Overall effect of social group categories (total “effects”)
   • Direct/Indirect effects (causal mediation)

3. Policies to address causes and/or remediate social differences in health. (Policy/Intervention)

It is clear that evidence on the effectiveness and cost-effectiveness of public health interventions is often missing. Sometimes this is because policies are insufficiently subjected to outcome evaluation, perhaps because it is assumed that they are mostly beneficial and any positive outcomes can be taken as read.

There is, for example, a wealth of aetiological evidence…However, it often appears to be difficult to translate this information into new interventions and even when the interventions are implemented, their evaluation is often problematic.

…a large part of the literature is descriptive rather than analytical.

We found no support for the notion that the methods used to reduce smoking decrease inequalities in health between educational groups.

We want to know:

• Did the program work? If so,for whom? If not, why not?

• If we implement the program elsewhere, should we expect the same result?

• Did it decrease inequalities?

• If we aren’t controlling treatment assignment, who is?

• Policy programs rarely select people to treat at random.

  • Targeting those most likely to benefit.
    
  • Programs implemented in response to events.
    
  • State taxes (or subsidies) for certain goods.

• People do not choose to participate in programs at random.

  • Welfare programs, health screening programs, etc.
  • People who believe they are likely to benefit from the program.

Without randomization \((Z)\), we focus on exploiting:

• Treated group with change in exposure.
• Comparison group without change.

Recall the potential outcomes framework. We need a substitute population (treated and controls):

\[E[Y^{1}-Y^{0}]=E[Y^{1}|T=1]-E[Y^{0}|T=0]\]

• Where should we get our counterfactual?

The simplest DD setting:

• Outcomes observed for “units” observed in one of two groups:

  • Treated
  • Control

• Outcomes observed in one of two time periods:

  • Before intervention
  • After intervention.

• Treated: only units in one of the two groups are exposed to a treatment, in the second time period.

• Control: Never observed to be exposed to the treatment.

• The average change over time in the non-exposed (control) group is subtracted from the change over time in the exposed (treatment) group.

• Double differencing removes biases in second period comparisons between the treatment and control group that could result from:

  • Fixed (i.e., non time-varying) differences between those groups.
  • Comparisons over time in the treatment group that could be the result of time trends unrelated to the treatment.

• Basic DD controls for any time invariant characteristics of both treated and control groups.

• Does not control for any time-varying characteristics.

• If another policy/intervention occurs in the treated (or control) group at the same time as the intervention, we cannot cleanly identify the effect of the program.

• DD main assumption: in the absence of the intervention treated and control groups would have displayed similar trends.

• This is called the parallel trends assumption.

Classic epidemiology example: Water and cholera

• Couldn’t randomize.

• Lambeth moved intake upstream of London after 1849.

• SV similar to Lambeth, but did not move.

• SV as ‘unaffected’ control.

• Did not estimate DD parameter, but idea was there.

• Evidence of pre-treatment equivalence between groups:

“In many cases a single house has a supply different from that on either side. Each company supplies both rich and poor, both large houses and small; there is no difference either in the condition or occupation of the persons receiving the water of the different companies…”

• No knowledge of mechanisms or intervention:

“divided into two groups without their choice, and, in most cases, without their knowledge”

Let \(\mu_{it}=E(Y_{it})\)

• \(i=0\) is control, \(i=1\) treated.

• \(t=0\) is pre, \(t=1\) is post.

• One ‘difference’ is pre-post in treated: \(\mu_{11}-\mu_{10}\)

• Second ‘difference’ is pre-post in control: \(\mu_{01}-\mu_{00}\)

• Differences-in-Differences: \((\mu_{11}-\mu_{10})-(\mu_{01}-\mu_{00})\)

Snow’s Example:

Single treated and control group, two periods:

• \(\beta_{1}\) = Treated group

• \(\beta_{2}\) = Post period

• \(\beta_{3}\) = Product term

\[Y = \color{blue}{\beta_{0} + \beta_{1}*treat} + \color{red}{\beta_{2}*post} + \color{green}{\beta_{3}*treat*post}\]

How to estimate the impact?

• \(Treat=1\) if Lambeth, 0 if SW

• \(Post=1\) if 1854, 0 if 1849

• \(Treat*Post=1\) if Lambeth in 1854, 0 otherwise.

\(Y=\beta_{0}+\beta_{1}Treat+\beta_{2}Post+\beta_{3}Treat*Post+\varepsilon\)

Express our earlier model using ‘fixed effects’:

• Dummy for Group

• Dummy for Time

• Time-varying policy indicator

\[Y = \color{blue}{\beta_{0} + \beta_{1}*Group2} + \color{red}{\beta_{2}*Time2} + \color{green}{\beta_{3}*policy}\]

\(\color{green}{\beta_{3}}\) still estimates the ‘difference-in-differences’ parameter.

What about multiple treated groups?

• Easy to rewrite our earlier model for multiple groups treated at the same time.

• 3 units and 3 time periods.

• Groups 1 and 3 implement policy at T2.

• g2 and g3 are dummies for group 2 and 3

• t2 and t3 are respective time dummies for periods 2 and 3.

We want credible counterfactuals

• Strong visual evidence that pre-intervention trends similar in treated and control groups.

• Adds credibility to assumption that post-intervention trends would have been similar in the absence of the intervention.

National policies in 1999

Tackling low income, family support policies, tax-reduction and long-term care for the elderly, anti-smoking policies, improving early education

National policies in 2003

Reducing poor health behaviors in manual social groups, improving housing quality, and reducing accidents at home and on the road.

Data on self-reported health, smoking, obesity

In the last and our main step, we added each of the comparison countries separately to the analysis of the English data, following the idea of “difference-in-differences analysis”. Our aim was to investigate whether the changes in trends in health inequalities between 1990–2000 and 2000–2010 were more favourable in England than those in the three comparison countries.

• Socioeconomic differences may be different magnitude in treated vs. control areas.

• Better resources, more advocacy, different demographics, etc.

• Secular trends may be changing differentially by social group in all areas.

• Different baseline health, health behaviors, access to resources, etc.

The interpretation of the interaction terms in difference-in-differences logistic models is essentially similar to that in the more common linear models, except that they indicate the relative change of the odds of the health outcome in the treatment group relative to that in the control group, instead of the absolute change of the rate of the health outcome in the treatment group minus that in the control group

• Parallel trends assumption is scale dependent.

• Can’t have it both ways.

Parallel trends in levels (\(\Delta 0.8\)) consistent with positive impact of treatment.

Parallel trends in logs (\(\times 1.3\)) consistent with negative impact of treatment.

Note that our basic regression model assumes the only time-varying factor is the policy: \[Y_{gt}=\alpha + \gamma_{g} + \tau_{t} + \color{red}{\delta^{DD}} \color{black}{p_{gt}} + \varepsilon_{gt}\]

What if there are confounders of the decision to change the policy?

We may have omitted important factors that:

• differ by treatment status.
• affect the outcome.
• are time-varying, but not affected by the treatment \((TVC)\).

DD design can also handle treatments, policies, or exposures that are not dichotomous.

• E.g., changes in minimum wage levels (varying “treatment” intensity)

  • Estimate varying levels of increase ($2 increase vs. $4)

• “Sin” taxes (e.g., alcohol or cigarettes).

  • differential changes in excise taxes (smaller vs. larger).

• “Weaker” vs. “Stronger” policies

  • texting while driving (primary vs. secondary offense)

  • thresholds for blood alcohol limits (0.15 vs. 0.10 vs. 0.08).

• Basic DD estimates the average ATT over the entire post-intervention period.

• May average over important variations in how the treatment evolves over time.

• Was the impact immediate? Transient? Sustained over time?

• Can extend the basic model to allow for heterogeneity over time.

• Different groups adopt treatments at different times.

• Creates many 2x2 DDs.

1. Early-adopters (k) vs. never treated (U)

2. Later-adopters (l) vs. never treated (U).

3. Early (k) vs. later (l) adopters.

4. Later (l) vs. earlier (k) adopters.

• Using earlier treated groups as controls only ‘works’ if the treatment effects are:

  • Homogeneous across groups at a given time; and
  • Homogeneous over time (no dynamic effects).

• This adds any changes in treatment effects in the early group, which get subtracted from the DD estimate.

• Can lead to \(\beta^{DD}\) that is a poor summary of group-specific effects if there is heterogeneity.

Key Takeaways

• DD compares changes in outcomes in a treated group to a control group.

• Controls for time-invariant unobserved group factors and common trends in outcomes.

• Requires good qualitative knowledge about why the treated group became treated.

• Core assumption is parallel trends, unverifiable but not impossible to investigate.

• Can be extended to address inequalities, but stronger assumptions needed.

• Strong designs like DD can help reduce the “evidence gap”.